A machine-learning based model for automated recommendation of individualized treatment of rifampicin-resistant tuberculosis

Background Rifampicin resistant tuberculosis remains a global health problem with almost half a million new cases annually. In high-income countries patients empirically start a standardized treatment regimen, followed by an individualized regimen guided by drug susceptibility test (DST) results. In most settings, DST information is not available or is limited to isoniazid and fluoroquinolones. Whole genome sequencing could more accurately guide individualized treatment as the full drug resistance profile is obtained with a single test. Whole genome sequencing has not reached its full potential for patient care, in part due to the complexity of translating a resistance profile into the most effective individualized regimen. Methods We developed a treatment recommender clinical decision support system (CDSS) and an accompanying web application for user-friendly recommendation of the optimal individualized treatment regimen to a clinician. Results Following expert stakeholder meetings and literature review, nine drug features and 14 treatment regimen features were identified and quantified. Using machine learning, a model was developed to predict the optimal treatment regimen based on a training set of 3895 treatment regimen-expert feedback pairs. The acceptability of the treatment recommender CDSS was assessed as part of a clinical trial and in a routine care setting. Within the clinical trial setting, all patients received the CDSS recommended treatment. In 8 of 20 cases, the initial recommendation was recomputed because of stock out, clinical contra-indication or toxicity. In routine care setting, physicians rejected the treatment recommendation in 7 out of 15 cases because it deviated from the national TB treatment guidelines. A survey indicated that the treatment recommender CDSS is easy to use and useful in clinical practice but requires digital infrastructure support and training. Conclusions Our findings suggest that global implementation of the novel treatment recommender CDSS holds the potential to improve treatment outcomes of rifampicin resistant tuberculosis.


Introduction
Tuberculosis (TB) continues to be a global public health problem with about 10 million new cases and 1.4 million TB deaths annually [1]. The 'End TB Strategy' of the World Health Organization (WHO) aims to reduce new TB cases by 90% and TB deaths by 95% by 2035. A major challenge to achieve these goals is the occurrence of about 450,000 new cases of rifampicin resistant TB (RR-TB) annually [1], cases which are di cult, complex, and costly to treat.
All patients diagnosed with RR-TB should start a standard RR-TB treatment regimen. Since 2019, a 9-12 month all-oral 7drug RR-TB regimen is recommended for patients who have not been previously treated for ≥ 1 month with bedaquiline, clofazimine or linezolid and in whom resistance to uoroquinolones is unlikely or excluded [2]. Otherwise, a longer (18 to 20 months) regimen containing all three group A agents (levo oxacin/moxi oxacin, bedaquiline, linezolid) and at least one group B agent (clofazimine, cycloserine/terizidone) should be administered. Group C agents (ethambutol, delamanid, pyrazinamide, imipenem-cilastatin/meropenem, amikacin/streptomycin, ethionamide, prothionamide, para-aminosalicylic acid) can be added to ensure the regimen contains at least four drugs to which the Mycobacterium tuberculosis (Mtb) strain is most likely susceptible [2]. In 2022, a 6-month regimen containing bedaquiline, pretomanid, linezolid, and moxi oxacin (BPaLM) was endorsed by the World Health Organization (WHO) for treatment of RR-TB and a 6-9 month regimen of bedaquiline, pretomanid, and linezolid (BPaL) for uoroquinolone resistant RR-TB [3], but pretomanid is not yet registered or available in most countries.
Ideally, the standard RR-TB treatment regimen should be individualized when drug susceptibility test (DST) results identify the presence of resistance to one or more drugs included in the regimen. Because DST assays can take months, this is not always implemented. For example, between November 2012 and December 2013 treatment was individualized in only one in ve (21.0%) South African RR-TB patients [4]. Molecular methods are increasingly used for rapid DST [5]. Line probe assays (LPA) are used in many countries for detection of resistance to isoniazid, injectable drugs and uoroquinolones in people diagnosed with RR-TB. More recently, the Xpert MTB/XDR assay for the diagnosis of resistance to isoniazid, uoroquinolones, ethionamide, and injectable drugs and a LPA for detection of resistance to pyrazinamide have been endorsed by the WHO but these are not yet implemented in most countries [6]. Rapid tests for ethambutol, terizidone, cycloserine, bedaquiline, clofazimine, linezolid, delamanid, pretomanid and PAS do not exist. In the absence of a complete drug resistance pro le, it is challenging to correctly compose an individualized RR-TB treatment regimen.
Next generation sequencing (NGS), using a targeted or whole genome sequencing (WGS) approach, can provide a comprehensive genomic drug resistance pro le. In 2018, the WHO endorsed NGS for surveillance but not for clinical care [7]. Some public health laboratories in high income, low RR-TB burden settings have integrated WGS into the routine patient management [8][9][10]. In low-and middle-income countries, the use of NGS remains limited to research institutions and reference laboratories [11] because of limited bioinformatics expertise, scarce sequencing infrastructure, expertise required for the interpretation of sequencing data, and challenges in translating a drug resistance pro le into the optimal individualized treatment regimen [12,13]. Increased automation could facilitate the use of NGS data for individualized RR-TB treatment and increase the patient bene t that could arise from the scienti c advances made.
In this paper, we describe the development of a computational model using machine learning methods to create a clinical decision support system (CDSS) that automates the translation of NGS data into an individualized RR-TB treatment regimen recommendation with its accompanying user-friendly web interface and present the results of the acceptance assessment of the tool by physicians in South Africa.

Development of the RR-TB treatment recommender CDSS
The development process of the RR-TB treatment recommender CDSS consisted of the assembly of the knowledge base, development of a heuristic model prototype, feedback harvesting from experts, application of machine learning methods to analyze the feedback, assessment of the CDSS performance, and external validation [14]. The stakeholder group included experts in pathogen genomics (genotype-phenotype associations), pharmacology (drug properties, mechanisms of action, drug-drug interactions, synergy and antagonism between drugs), medicine (treatment of RR-TB in high and low burden settings), and computer and data science. In addition, public health practitioners and patients were included to hear their viewpoints. Stakeholder discussions were held in 2019 and focused on the minimum number of effective drugs to be included in a regimen, the role of an intensive treatment phase, drug toxicity, burden of treatment monitoring, drug properties that de ne effectiveness, and level of drug resistance.
The key features of individual drugs and treatment regimens were quanti ed by ve experts in 2019, through review of published and unpublished data. In the heuristic model prototype all features received equal weight, possible interaction between features were not considered, and all regimen features were normalized to obtain a regimen score with a higher score representing a better regimen.
A training dataset of 129 unique WGS-derived drug resistance pro les of which 119 (92%) were resistant to rifampicin, 106 (82%) to isoniazid, and 29 (22%) to uoroquinolones from 303 South African RR-TB patients was used to solicit feedback from experts. For each unique drug resistance pro le, all available four-drug regimens (de ned as a regimen not containing any drug to which the Mtb isolate is resistant on WGS) were generated by the treatment recommender model. A sampling function was used to increase the likelihood for feedback on highly ranked (i.e., better) compared to lower ranked regimens. Regimens were replaced to ensure that multiple experts could provide feedback on the same regimen. Experts were asked to state if they would prescribe the proposed regimen to a patient with the speci ed resistance pro le. If they rejected the regimen, the reason for rejection and suggestion of an alternative regimen was requested. After the rst round of feedback, experts discussed which features were missing, obsolete, or sub-optimally quanti ed and revisions to the model were made. After completion of each feedback round, a random forest classi er machine learning model was trained on the expert feedback to determine the importance of the different drug and regimen features and the interactions between these features. The result of the machine learning analysis was then used to develop the next version of the treatment recommender model [15]. The iterative feedback harvesting process was continued until no further signi cant improvements were made in the model prediction of a good treatment for a speci ed resistance pro le. The model was then validated using a dataset of 64 unique WGS-derived drug resistance pro les that were not present in the training dataset [14].

Development of interactive online web interface
We developed a user-friendly web application optimized for mobile phone (available from https://github.com/LennertVerboven/treatment_recommender_webapp). The front end was written in JavaScript using the React library. The back end written in Python3, using the Django web development framework, exposes a representational state transfer (REST) application programming interface (API) to which the clients can make secured hypertext transfer protocol secure (HTTPS) requests. The front end allows health care workers to log in securely using username and password authentication and then shows a list of their patients. For each patient, the app lists the patient's demographic information (name, date of birth, clinic, phone number), current treatment regimen, contra-indications based on clinical information, toxicity, stockouts, the WGS-based resistance pro le, and the recommended regimen. The care provider can use the app to request an updated individualized treatment regimen in real-time, in case of toxicity or when a drug is out of stock.
Acceptance assessment of the RR-TB treatment recommender CDSS The treatment recommender CDSS and its accompanying webapp were implemented as part of the SMARTT pragmatic trial (Clinicaltrials.gov Identi er NCT05017324) [16]. The trial aimed to determine the effectiveness of a WGS-based drug susceptibility testing strategy to guide individualized treatment for patients diagnosed with RR-TB.
Acceptance of the treatment recommender and its accompanying online app by clinicians caring for trial participants was assessed in three different ways. First, we investigated the proportion of trial participants for whom the treatment proposed by the treatment recommender CDSS was prescribed. Because this is an ongoing trial, we performed this analysis on the rst 20 patients randomized to the intervention (WGS) arm. Second, we determined the proportion of patients for whom clinicians would prescribe the regimens proposed by the treatment recommender in their routine practice, i.e., outside of the trial setting. This was done by presenting the treatment regimens recommended by the CDSS for 15 unique resistance pro les to ve physicians who had experience with the CDSS in the trial setting and agreed to participate in this assessment. Regimens were selected from those that have been documented in the region where these physicians practice. Physicians were asked if they would prescribe the proposed regimen for a patient receiving routine RR-TB care.
Third, we performed a survey to assess the factors that may in uence health care workers' use of the treatment recommender CDSS. The questionnaire (S1 table) was developed based on the modi ed technology acceptance model ( Fig. 1) and consisted of 36 questions investigating eight domains within the technological, individual, and organizational context (Table 1) [17]. Participants were asked to rate each question on a 5-point Likert scale. Responses were transformed into a score ranging from 0 (Strongly disagree) to 4 (strongly agree), except for questions 8 and 32 for which the rating was inverted as these questions were posed negatively. For each domain, the median and range of the sum of the scores were calculated. Table 1 The eight domains of the technology acceptance model and their de nitions [17] Dimension De nition

Perceived usefulness
Belief that the treatment recommender CDSS will help them care better for patients with RR-TB

Perceived ease of use
The degree to which the physician believes that using the RR-TB treatment recommender CDSS would be effortless Attitude Perception of the positive or negative consequences related to adopting the treatment recommender CDSS

Compatibility
The degree of correspondence between the treatment recommender CDSS and existing values, past experiences and needs of potential adopters

Subjective norm
The extent to which an individual believes that people who are important to him or her will approve the physician's adoption of the treatment recommender CDSS

Facilitators
The degree to which an individual believes that the organizational and technical infrastructure required to support the use of the treatment recommender CDSS exists

Habit
The degree to which the use of the treatment recommender affects habits and behaviors that have become automated

Intention
Willingness of the physician to use the treatment recommender CDSS if it becomes available

Selection and quanti cation of individual drug features
The drug features identi ed by the stakeholders were toxicity, early bactericidal activity (EBA), bactericidal activity, sterilizing activity, mode of administration (oral vs injection), mechanism of drug action, propensity to acquire resistance, and cost (S2 table). After the rst round of feedback, QTc prolongation was added as a separate feature, resulting in nine features to characterize individual drugs in the model. Toxicity was classi ed for each drug as life threatening, permanent (e.g., hearing loss), short-term with possible effect on adherence (e.g., nausea) or minimal without patient impact (e.g., liver function test abnormality grade 2). Based on a combination of frequency of occurrence and severity, the level of toxicity for each drug was graded on a scale of 1 to 3. QTc prolongation was classi ed into four categories (none, low, moderate, or high). Early bactericidal activity (EBA), bactericidal activity and sterilizing activity were classi ed as one of ve categories (very low, low, moderate, high, very high). EBA classi cation was based on the drop in log 10 colony forming units (CFU) in the rst days of treatment observed in experimental EBA studies and bactericidal activity on the drop in log 10 CFU in the rst six months of treatment. Sterilizing activity, or the ability to achieve stable cure was de ned based on a drug's ability to prevent relapse in human or animal model studies. The propensity to acquire resistance was classi ed into four categories (low, moderate, high, and none) based on expert opinion. Mode of administration was de ned as either oral or injection (intravenous or intramuscular). All cost for the drugs at recommended dosing was quanti ed as the cost for one month of treatment in South Africa [18][19][20][21].
The uoroquinolones, moxi oxacin and levo oxacin are oral drugs that inhibit DNA gyrase of Mtb [22]. Fluoroquinolone toxicity was classi ed as low (score 1 for levo oxacin, 1.25 for moxi oxacin) given the low (1.2%) incidence of serious adverse effects (SAE) in RR-TB patients receiving uoroquinolones [23]. QTc prolongation on moxi oxacin is moderate [24] and experts classi ed the risk of QTc prolongation for levo oxacin as low. The monthly cost is $3.87 for levo oxacin and $8.85 for moxi oxacin [21]. The EBA of uoroquinolones is very high based on a fall in log 10 CFU of 0.45 for moxi oxacin and 0.53 for levo oxacin [25]. Experts classi ed the bactericidal activity as moderate for levo oxacin and high for moxi oxacin. The sterilizing activity of uoroquinolones was judged to be moderate [26][27][28]. Experts determined that uoroquinolones have a low propensity to acquire resistance.
Rifabutin is an oral drug that kills mycobacteria by inhibiting the Mtb RNA polymerase [29]. Toxicity was classi ed as low (score 1.5) [30]. Experts stated that rifabutin does not prolong the QTc interval. The cost of 1 month treatment is $57.6 [21].
The EBA of rifabutin is low (fall in log 10 CFU 0. 0.07 [31]) and sterilizing activity is very high [32]. Experts classi ed the bactericidal activity as moderate. Rifabutin has a low propensity to acquire resistance.
Bedaquiline is an oral drug that inhibits the Mtb ATP synthesis [33]. Experts classi ed the toxicity for bedaquiline as low (score 1). The risk of QTc interval prolongation during bedaquiline treatment is moderate. Cost of one month bedaquiline treatment is $65 [21]. The EBA of bedaquiline is low with a fall in log 10 CFU of 0.04 [34] in the rst two days of a loading dose of 400mg/day. Experts classi ed the bactericidal activity as moderate and sterilizing activity as very high.
Bedaquiline has a moderate propensity to acquire resistance.
Clofazimine is an oral drug that inhibits protein synthesis [35]. Experts classi ed the toxicity of clofazimine as relatively high (score 2.25), it has no effect on the QTc interval [36]. The monthly cost of clofazimine treatment is $29.96 [21].
Clofazimine has a very low EBA as clofazimine treatment does not result in a drop in CFU counts in the rst week of treatment [37]. Experts classi ed the bactericidal activity as moderate and sterilizing activity as low, and it has a low propensity to acquire resistance.
Linezolid is an oral drug that inhibits protein synthesis [38]. The risk of toxicity caused by linezolid is high (score 3) [39]. Expert opinion stated that linezolid has no effect on the QTc interval. The cost is $42.7 per month of linezolid treatment [21], it has a moderate EBA with a 0.17 drop in log 10 CFU in the rst two days of treatment [25]. Experts classi ed the bactericidal and sterilizing activity of linezolid as high and the propensity to acquire resistance as low.
Ethambutol is an oral rst line drug that inhibits the Mtb cell wall synthesis [40]. Toxicity is low (score 1), with a risk of SAE of 0.5% [23]. Expert opinion stated that ethambutol has no effect on the QTc interval. One month of ethambutol treatment cost $2.19 [21]. Ethambutol has a low EBA with a 0.245 drop in log 10 CFU in the rst two days of treatment [41]. Experts classi ed the bactericidal activity as moderate, and it has almost no sterilizing activity [32] and a moderate propensity to acquire resistance.
Pyrazinamide is an oral drug that disrupts plasma membranes [42]. Toxicity of pyrazinamide is low (score 1) based on a 2.8% risk of SAE on pyrazinamide treatment [23]. Experts judged that pyrazinamide treatment has no effect on the QT interval. One month of treatment cost $2 [21]. Pyrazinamide has very low EBA [41,43] and very high sterilizing activity [32]. Experts classi ed the bactericidal activity as moderate. Pyrazinamide has a high propensity to acquire resistance.
Isoniazid is an oral drug that inhibits Mtb mycolic acid synthesis [44]. Experts classi ed the toxicity as low (score 1) for isoniazid and high-dose isoniazid (score 1.25). Experts judged that isoniazid has no effect on QTc interval. One month of treatment cost $0.61 for isoniazid and $1.22 for high-dose isoniazid [21]. Isoniazid has a very high EBA with a drop in log 10 CFU of 0.50 [32] and moderate sterilizing activity [32]. Experts classi ed the EBA of high-dose isoniazid also as high and viewed the bactericidal activity of isoniazid and high-dose isoniazid as high. It has a low propensity to acquire resistance.
The thioamides, ethionamide and prothionamide, are oral drugs that inhibit Mtb cell wall synthesis [45]. Thioamides have a high toxicity (score 3) with a 8.2% SAE risk [23]. Experts judged that thioamides do not in uence the QTc interval. The cost of one month of treatment is $11.8 for ethionamide and $18.95 for prothionamide [21]. Experts classi ed the EBA and bactericidal activity of thioamides as moderate, the sterilizing activity as low. Thioamides have a moderate propensity to acquire resistance.
Carbapenems, imipenem-cilastatin and meropenem, are administered via daily infusion together with oral clavulanate. They inhibit peptidoglycan synthesis [46]. Experts classi ed carbapenems as moderately toxic (score 1.75). Experts stated that carbapenems do not affect the QTc interval. Carbapenems are expensive drugs with a cost of $439.2 for one month imipenem and $338.55 for one-month meropenem [21]. The EBA is very low with small increase in log 10 CFU in the rst 14 days of imipenem treatment [47]. Experts classi ed carbapenems as having a low bactericidal activity. In combination with clavulanate, carbapenems have a high sterilizing activity [48]. Carbapenems have a low propensity to acquire resistance.
The aminoglycosides (amikacin, kanamycin, and capreomycin) and streptomycin, drugs which need to be administered by daily intramuscular injection, inhibit protein synthesis [49]. Of the aminoglycosides only amikacin is currently recommended for treatment of RR-TB. Amikacin and streptomycin treatment is highly toxic (score 3) with 7.3% of patients reporting SAE for both injectables [23]. Experts judged that amikacin and streptomycin have no effect on the QTc interval.
The EBA of the injectable drugs is low, with a loss of log 10 CFU of 0.05 for amikacin [25] and 0.04 for streptomycin [25]. The bactericidal activity and sterilizing activity of amikacin and streptomycin are also low [32,50]. Both amikacin and streptomycin have a low propensity to acquire resistance.
Cycloserine and terizidone are oral drugs that inhibit protein synthesis [51]. These drugs have a high level of toxicity (score 2.75 for cycloserine and 3 for terizidone with a 4.5% and 9.1% SAE risk in patients treated with cycloserine and terizidone, respectively) [23,52]. Experts stated that these drugs do not in uence the QTc interval. The cost of one month treatment is $32.7 for cycloserine and $219.6 for terizidone [21]. Experts classi ed the EBA, bactericidal and sterilizing activity as low, moderate, and low for both drugs respectively. These drugs have a moderate propensity to acquire resistance.
Para-aminosalicylic acid (PAS) is an oral drug that inhibits DNA precursor synthesis [53]. PAS treatment is highly toxic (score 3) as 12.2% of patients treated with PAS reporting SAE [23]. There is no data to suggest that PAS in uences QT interval. One month of PAS treatment costs $122 [21]. PAS has a moderate EBA with a drop in log 10 CFU of 0.259 in the rst two days of treatment [43]. Experts classi ed the bactericidal activity of PAS as low and its sterilizing activity as moderate. PAS has a moderate propensity to acquire resistance.
The nitroimidazo-oxazoles, delamanid and pretomanid are oral drugs that inhibit mycolic acid synthesis [54]. At time of the development of the treatment recommender, only delamanid was registered for RR-TB treatment in South Africa. Experts classi ed toxicity of delamanid as low (score 1). QT prolongation is also low [55]. One month of delamanid treatment costs $308.63 [21]. Delamanid has a low EBA (drop in log 10 CFU of 0.066 [25]), a low bacterial and moderate sterilizing with dose-dependent killing rates. At the highest dose of delamanid, its sterilizing activity is superior to isoniazid and equal to rifampicin [56]. Delamanid has a moderate propensity to acquire resistance.

Selection and quanti cation of regimen features
Experts agreed that an effective treatment regimen should consist of four effective drugs i.e., drugs to which no resistance has been detected. Fourteen features were de ned to characterize the treatment regimens in the model. Features one to six are the sum of features of the four drugs included in the regimen: the sum of EBA, bactericidal activity, sterilizing activity, toxicity, propensity to acquire resistance and cost. Feature seven to nine assess whether the regimen adheres to the principles to construct a TB treatment regimen developed by Van Deun et al. [57] which state that a regimen should contain 1) at least one core drug that has a high bactericidal and sterilizing activity, 2) at least one highly bactericidal companion drug, one moderately bactericidal companion drug, and two moderately sterilizing companion drugs (a single companion drug can satisfy multiple requirements) and 3) a combination of a core and companion drugs. Features 10 and 11 quantify either the number of interactions between drugs in the regimen synergistic (pyrazinamide and bedaquiline, pyrazinamide and clofazimine, pyrazinamide and delamanid [58, 59]) or antagonistic (moxi oxacin and rifampicin, bedaquiline and rifampicin [60, 61]). Feature 12 checks whether all four drugs have a different mechanism of action. Given the current goal to preferentially administer all-oral regimens, feature 13 indicates whether the regimen is an all-oral regimen. Because experts rejected regimens containing two or more drugs with a moderate to high QTc prolongation effect during feedback, feature 14 was added to evaluate whether the sum (with low, moderate, and high equaling 1, 2, 3 respectively) of the QTc prolongation of the drugs is greater than 3.
Binary features are directly used as input for the treatment recommender model, while the continuous features undergo two transformations. Each continuous feature is therefore represented twice in the nal treatment regimen feature set. The continuous features are normalized from 0 to 1, where the regimens with the highest score and lowest score obtain a value of 1 and 0 for that feature respectively. The second transformation divides the feature range into four parts (very high, high, low, very low) and classi es each continuous feature withing these classes. Negative features, such as cost, toxicity, antagonism, and propensity to acquire resistance are inverted such that higher values always equal patient bene t. Given that the total range for features varies when regimens are excluded (due to resistance), these transformations are performed after eliminating regimens containing drugs to which resistance was detected. Table 2 shows the 14 features (only showing the continuous features before transformation) of the 10 highest ranked regimens by the CDSS for a patient infected with a Mtb strain that is resistant to rifampicin, isoniazid, ethambutol, ethionamide, prothionamide, rifabutin, and streptomycin. The regimen ranked number 1 would be recommended, the other nine regimens ranked in the top 10 for this resistance pro le are increasingly suboptimal treatment regimens and the 100th ranked regimen is an indication of a poor regimen for this resistance pro le. The recommendation probability is the probability that the treatment recommender would classify this regimen as appropriate.    Table containing the regimen features for the rst 10 and the 100th ranked regimen for a Mtb strain resistant to  rifampicin, isoniazid, ethambutol, ethionamide,  Of the rst 20 patients enrolled in the SMARTT trial for which the treatment recommender was used to recommend the individualized treatment regimen, all 20 (100%) patients received the regimen recommended by the treatment recommender CDSS when taking the Mtb resistance pro le, clinical data of the patient and drug availability at the clinic into account. For 7 (35%) patients, the initial recommendation had to be adjusted because of the presence of a clinical contra-indication for one of the drugs (n = 3) or because one of the drugs included in the recommended regimen was not available at the clinic (n = 4), information which was not available when the initial regimen was computed. For these seven patients the treatment recommender CDSS was rerun after the physician provided this information, and these patients were prescribed the recomputed treatment regimen (Fig. 2). For one patient, the regimen had to be recomputed at week 8 of individualized treatment due to a serious adverse effect to linezolid.
Acceptance of treatment recommender CDSS in a routine care setting All physicians agreed to prescribe the recommended regimen for 4 of the 15 pro les, and all physicians disagreed with 2 of the 15 pro les ( Table 3). The reason for most rejections were deviations from national RR-TB treatment guidelines. The most common reasons were that the South African guidelines state that bedaquiline should not be combined with moxi oxacin and that group A drugs should always be combined with group B drugs. In addition, a regimen was rejected because the physician believed that resistance to isoniazid always occurs in case of ethionamide resistance, which is not the case. Acceptance of the treatment recommender CDSS system and its accompanying webapp When using the modi ed technology acceptance model as a guide to assess the acceptance of the treatment recommender CDSS and its accompanying webapp (S1 table), we found that the score was high (≥ 75%) for most domains except for the subjective norm (67%) and facilitators domain (58%) ( Table 4). The facilitators domain scored low because physicians stated that their settings did not have the required infrastructure. Internet connection or access to email is not always available in more rural settings and they often use personal devices that do not have infrastructure support. The full questionnaire with answers is available in S1

Discussion
When a patient is infected with an Mtb strain that is resistant to one or more of the drugs included in the standardized treatment regimen, has a contra-indication to one of the drugs, or experiences a side effect that requires one of the drugs to be stopped either temporarily or permanently, an individualized treatment should be initiated. Using a combination of machine learning and expert knowledge, we successfully developed a CDSS that automatically composes an individualized treatment that balances effectiveness, tolerability from a patient perspective, and feasibility from a health system perspective. The CDSS facilitates decentralized care, even for those RR-TB cases which are not eligible for a standard RR-TB regimen because of drug resistance, contra-indication or toxicity to one of the drugs in the standard regimen. The interactive online tool further maximizes the utility of the CDSS as it allows physicians to enter patient characteristics (toxicity or contra-indications for certain drugs) and relevant health system characteristics (registration of certain drugs in a speci c country or temporary drug stock outs at a certain facility) in real time. The system also generates a pdf document that can be printed for paper record keeping. The report can be modi ed such that it is user-friendly, intuitive, and useful for health care workers with different levels of knowledge of genome sequencing or experience in treating DR-TB.
The acceptability of the CDSS was assessed in the context of the SMARTT clinical trial, where we found that, among the rst 20 trial participants for whom the CDSS was used, all patients started a regimen recommended by the CDSS. In most (65%) trial patients the regimen recommended solely based on the WGS-derived drug resistance pro le was started. In the other third of patients, the CDSS had to be run a second time because one of the drugs included in the regimen was out of stock, presence of a clinical contra-indication or development of toxicity. In such cases, the webapp can be used to re-run the model in real time, while the patient is with the physician. When assessing the acceptability of the treatment recommender outside of the trial setting, we found that physicians were hesitant to deviate from the guidelines.
Implementation of the CDSS in routine care will thus need to be accompanied by updated guidelines to re ect new knowledge, such as the ability to safely combine moxi oxacin and bedaquiline, and to correct misconceptions, such as the belief that ethionamide resistance is always associated with isoniazid resistance. From a digital technology perspective, acceptance of the treatment recommender CDSS and its app was high, with physicians indicating that they believe the treatment recommender is both easy to use and useful in clinical practice. The main concern for successful implementation in clinical practice was the infrastructure and training required. drug. Re-training the entire model is only needed when a new feature is added. Furthermore, when, in future, the treatment recommender would be used on a large scale, the treatment outcome of patients receiving CDSS-guided could be used to re-train and iteratively improve the model over time.
Several limitations to the treatment recommender should also be noted. First, there is no 'truth' for what constitutes the optimal individualized DR-TB treatment regimen, making it di cult to assess the performance of the treatment recommender. Second, there may be patient-or pathogen-related considerations, such as extent and type of disease, that are not yet fully captured in the model. The treatment recommender should thus be viewed as a CDSS tool and not a substitute for clinical judgement. Third, the knowledge base was developed using published and unpublished data available in 2019 complemented with expert opinion when data was scarce. Updating the feature values with new knowledge could improve the accuracy of the model. Fourth, the price of the different drugs can vary between regions, countries, and over time, which may need regional adaption of the feature values. Finally, our study is limited to the South African setting, therefore prospective studies should demonstrate the effectiveness of the treatment recommender CDSS in other high burden settings.
In conclusion, the treatment recommender and its accompanying online platform present a novel strategy for real-time user-friendly support for decentralized management of treating complex RR-TB patients. Global implementation of such a treatment recommender CDSS can help realize the goal of prescribing the most effective and least toxic treatment regimen for all patients suffering from DR-TB.